posted 11-30-2007 08:29 PM            

stat:

quote:

---

NEVER EVER BE AFRAID TO CALL INCONCLUSIVE/ NO OPINION. The pressure to make a call will always be there----so learn to explain what it means---use plenty of analogies. I don't cvare what some would say, some formats used in our profession cause a higher level of inconclusives than others. No one likes inconclusives--least of all the examiner. Tell your superiors--in an impathetic way---that you don't like them either. The common knowledge is that newer examiners get more inconclusives----who knows precisely why. It's probably a number of reasons. A good rule of thumb is to not hype your tests with a pre-pretest drum roll to your authorities---because when the dreaded INCL arises---they tend to disappoint more. 2 cents

---

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 11-30-2007 10:09 PM            

------------------
"Truth Seekers"

posted 12-01-2007 11:07 AM            

These are great ideas, and from previous conversations I know that you have a lot more.

So here is my .02 on this complication.

When we test someone, our job is, in part, to assure that the examinee will react significantly to the stimulus if he is deceptive. It is also part of our role to conduct the examination in a manner so as to ensure that the examinee will react significantly to the stimulus for no other reason than being deceptive. There does seem to be a general consensus and theoretical rationale that a confrontational interrogation just before a polygraph test, could sensitize a truthful examinee to react significantly to the stimulus regardless of his truthfulness. There is also some theoretical rationale, empirical support, and general consensus (at least among us polygraph examiners), that truthful examinees will react significantly to the comparison questions (which we interpret at the absence of significant reaction to the relevant questions). However, the proposition that we can stack these constructs like Legos does not seem to be all that sound. There are numerous additional unknowns that occur when we first sensitize an examinee with a confrontational pre-polygraph interrogation and then attempt to desensitize the examinee with our narrative theme. As an analogy, we know that base substances will neutralize acids. We also know they don't simply cancel each other out in a completely inert manner, and the result of combining acids and bases is a salt. My point is that there is no reason assume that any interviewing theme reliably or predicatably mitigate the sensitization of a pre-polygraph interrogation.

I don't disagree that we cannot completely extinguish our awareness of the potential for errors. I would suggest that it is reasonable that we choose which hat we wear and which role we play. As a therapist, I am a mandatory child abuse reporter (and immune from civil liability); as a polygraph examiner, I am not a mandatory reporter (and also not immune from civil liability). I would guess that some police examiners change hats similarly, based on departmental policies perhaps, when they are authorized to conduct examinations outside their sworn law enforcement roles (for which an arrest might not be made in response to every felony admission.)

For the record: I have never done the robot dance - regardless of free beer (not that I haven't done other silly things, though).

r

(I wrote a little bit on the ethics of computer scoring while waiting for the temperatures to reach double digits so I could run around the lakes - I'll put that in a separate post.)

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 12-01-2007 11:30 AM            

quote:

---

If you use the Horizontal Scoring System, then sure, it takes a while (unless you use the algorithm), but you can hand-score as the charts are going by live or between charts.
Most consider it unethical to use the algorithm alone, so I'd run from that one.

---

Here is my .02 on the ethics of using computer scoring algorithms...

When we say this type of thing (unethical to use the computer scoring algorithm alone), its important to understand why say that type of thing. Otherwise hyperbole crystallizes into dogma and further thinking is extinguished.

First, we have to remember that professional ethics is often declarative – involving statements about what is, and is not, acceptable professional conduct. Those declarations are often in the form of standards of practice, which should be grounded in some form of principle related to the pursuit of some desired outcome. Outcomes, of course, can be considered positive or negative depending on your point of reference. (e.g., blowing one's enemies to tiny smithereens with a holy-hand-grenade is good, but having to contend with the holy-hand-grenades of others is bad.)

Concerns about over-utilizing the results of computer scoring algorithms are based on a couple of historical issues, including:

1. whether or not we know what physiological response features those scores are based upon,
   
2. whether or not we know how multiple physiological measurements are transformed mathematically into the single or mixed set of test questions (i.e., SR or NSR),
   
3. whether or not we understand the decision rules that parse those mathematical results into SR or NSR classifications, for single issue, multi-facet, and mixed-issues exams, and
   
4. whether we have any confidence in how anomalous and artifacted data are identified and excluded from the analysis.
   

There are perhaps other things that I'm not thinking of right now.

When hand-scoring polygraph charts we have a sense of answer to each of these concerns. We know what data features we attend to; we know how the data are combined (addition); we know our decision rules, and we know what data we consider un-interpretable.

It is generally considered unethical to render a professional opinion using a method which one does not understand or cannot articulate the empirical rationale and principles – or at the very least cannot provide sufficient references to others who might be interested in understanding the answers to the concerns listed above.

So, there is nothing inherently unethical about rendering a professional opinion using methods that meet the criteria above: describable (point-at-able) physiological features, documented mathematical transformations, clearly described decision rules per examination type, and an articulate understanding of how data integrity is evaluated or maintained – as liong as we understand how a result is achieved.

In the past, most computer scoring algorithms could not satisfy all of these concerns.

1. We have little or no documentation around the physiological features employed by some algorithms, and it is clear that some algorithms employ physiological features that are difficult for examiners to understand (try to envision the upper respiration data standardized to the 80th percentile).
   
2. Most, if not all, computer scoring algorithms in the past have provided only cursory descriptions of how the entire set of measurements from multiple test charts are combined and aggregated to a single set of values for the total and spot scores. They have also not provided complete descriptions of the distributions (norms, or normative models) of total and spot scores for truthful and deceptive subjects.
   
3. Most, if not all computer scoring algorithms in the past have have provided incomplete descriptions of the details of the decision rules used to parse the final results – especially pertaining to multi-facet and mixed-issues exams.
   
4. To be fair, CPS-II is based on three physiological features from the Utah system, which are easily understood by human examiners. Polyscore has included facilities for marking artifacted and uninterpretable data, and Axciton Chart Analysis does include the ability to remove uninterpretable questions from analysis. CPS-II includes features for editing and interpolation of uninterpretable data segments. OSS versions prior to the current one (version 3) were not tolerant of missing or uninterpretable data.
   

Another ethical guideline, pertaining to the rendering of any professional opinion is whether or not an opinion is based on a credible application of scientific principles, which means theoretical constructs and statistics. Determination of this requires complete documentation.

While hand-scoring systems readily satisfy concerns about physiological features, aggregation of data, coherence of decision rules, and exclusion of un-interpretable data, they remain subject to a remaining, though serious, limitations surrounding the final transformation of data to some form of probability value. In this respect, computer scoring algorithms are clearly superior, though would be possible to improve our hand-scoring methods in this area. At present it is not possible to compare point totals between the different hand-scoring system in any mathematically meaningful way – because we insist on describing those results in point totals with little understanding of their corresponding p-values (which would vary based on how the data are aggregated – via cumulative data models vs averaging and standardized models). I think Matte did provide some descriptives in his book (I'll check) regarding the distributions of truthful and deceptive scores, and those can be transformed to p-values – but that work is incomplete. Most other hand-scoring systems do not provide adequate descriptives and we have little understanding of the p-values represented by point totals (and those totals might differ based on the numbers of questions and charts when using cumulative aggregations). A particularly challenging problem exists around multi-facet and mixed-issues exams, because we would need to understand spot by spot confirmation in order to know the mean and deviations values for truthful and deceptive spots – a near impossibility with field data, though probably possible with a properly designed lab study, and entirely possible with a monte-carlo simulation (monte-carlos are really cool, and are used from everything from building bridges, to forecasting financials, planning battles, predicting football games, and creating video games). At present we cannot calculate p-values for spot scores using any of the existing hand-scoring systems – because we do not know the distribution of truthful and deceptive spots.

We are presently endeavoring to improve the general understanding of computer scoring systems among scientists and field examiners – by completely and openly documenting the operations of OSS-3 for any interested person to study. OSS-3 is based on simple physiological features that are easily recognizable to human examiners (the same features employed by CPS-II). OSS-3 addresses the challenge of understanding the distribution of spot scores through the standardization of those scores (to dimmentionless z-scores with a mean of zero and standard deviation of one) – so that OSS-3 can provide p-values for spot scores, using the same normal distribution as total scores.

OSS-3 can satisfy all of the concerns listed above: recognizable physiological features, clearly recognizable mathematical transformations, documented decision rules, and facilities for marking artifacted and un-interpretable data (thought that issue is actually addressed by the equipment manufacturers in their implementation of OSS-3 or any other algorithm). Plus, OSS-3 can do something which hand-scoring systems cannot yet do – provide a statistical p-value for spot scores.

Whether every field examiner in interested in or capable of understanding and describing the physiological and mathematical foundations of an algorithm is not relevant to the ethics of its use. What is important is adequate documentation, so that anyone who is interested or needs to understand or describe the results could do so.

So, if the ethical use of a scoring system is based on principles, and not merely on hyperbole, dogma, and appeal to authority, then I suggest that we are moving the science of polygraph forward to an era in which it may become less unethical (more ethical) to render professional opinions in response to computer scoring algorithm result, and simultaneously less ethical (more unethical) to render opinions on arcane systems that may be increasing regarded as statistically inadequate (though there are potential solutions to that limitation).

After all that I still hand score everything, and would suggest that we should always do so because it imposes another ethical point – that professionals are responsible for their opinions, and it is unethical to acquiesce or surrender professional judgment to the computer when making decisions that affect the lives of other human beings (even if they may be criminals).

Again, just my .02

Peace,

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 12-01-2007 06:31 PM            

First to the other Gordon graduate (Kieso) I would be glad to accept your offer for assistance. My email is on my profile.

Next, Stat. Great e-mail about post interrogation remedies. I don't want to say I do unethical tests because I will get booted from the profession, but when I am called in sometimes they want a call, but more then that they just want a confession. I will mention it to the Sgt. again about the post interrogation issue. I do love clearing suspects. I did that a couple of times this year.

Nelson, to the algorithms...when I do an algorithm I already have a pretty good idea of how its going to come out. You can't help but watch the test as its going on. I am not scoring as Barry does, but I am looking at the GSR and the Cardio. Especially the GSR. The pneumo's, I am not really comparing that during the test. Thats tough enough after the test.

To understanding the algorithm. I do understand Nates algorithm because I am trained in it. However his is different from the rest because you actually measure your own reactions so you can eliminate "bad" reactions. Polyscore? couldn't tell you how it works or even what it measures. I am caught red handed there.

Barry mentioned CIT's, I have done a couple and want to go over them again this week in class. They are tough to score. They are quite erratic and even though a subject passes them I am still not confident he is not involved. Do you have confidence in them? I have them; after I get e-mail set up I would love to pass them on.

Last I will give you a quote from a VERY well respected police examiner that may explain my trends.It's not exact...

"The poly is wrong up to 20% of the time. I can't take that chance so I figured the only way to know if someone is telling the truth or not is to get them to confess."

I won't give the name but he is a great guy, won several awards, and fills his interview classes. The reason I won't give his name is (a) in case I messed up the quote (b) It seems like it goes against what you guys are saying.

Another thought on post interrogation. I tested a sex crimes suspect that failed miserably and was only 17- there is no reason I couldn't get a confession. But I didn't. He was interrogated before even coming by an Aunt, Mother, Uncle etc. Wouldn't that be the same thing? I thought good controls that visibly shook him up would combat that?

OK, my Fiance wants attention, so I'm done the novel, but again the wisdom is greatly appreciated.

Oh- to the smelly subjects. I was wrestler in college so testing can't be worse then wrestling someone that smells like #####. I(and another cop) wrestled with someone on duty 11 years ago that I will have to leave you hanging with. Lets put it this way you don't want to know the rest of the story if you are eating soon.

Requested Definitions

Canned Test-
NSR-

I apologize if I missed that in your post or should have picked it up through context. I did read Nelson's and Stat's posts twice. It sounds like PHD material. LOl

posted 12-01-2007 09:20 PM            

"NSR" means "no significant responses." It corresponds to NDI. We seem to be moving toward that conclusion for multi-issue tests. We could talk all night about how to define "significant" in some of those tests.